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NECESSARY STOCHASTIC MAXIMUM PRINCIPLE FOR DISSIPATIVE 
SYSTEMS ON INFINITE TIME HORIZON 

CARLO ORRIERI AND PETR VEVERKA 


Abstract. We develop a necessary stochastic maximum principle for a finite-dimensional stochas¬ 
tic control problem in infinite horizon under a polynomial growth and joint monotonicity assumption 
on the coefficients. The second assumption generalizes the usual one in the sense that it is formu¬ 
lated as a joint condition for the drift and the diffusion term. The main difficulties concern the 
construction of the first and second order adjoint processes by solving backward equations on an 
unbounded time interval. The first adjoint process is characterized as a solution to a backward 
SDE, which is well-posed thanks to a duality argument. The second one can be defined via another 
duality relation written in terms of the Hamiltonian of the system and linearized state equation. 
Some known models verifying the joint monotonicity assumption are discussed as well. 


1. Introduction 


The study of the stochastic maximum principle (SMP in short) is a current field of research 
motivated by the interest in finding necessary (and sufficient) conditions for optimality for stochastic 
control problems. The general idea of the SMP consists in associating to every controlled trajectory 
an adjoint equation which is backward in time. Its solution, called a dual process (which is, in fact, 
a pair of processes), is shown to exist under some appropriate conditions and plays a role of 
“generalised Lagrange multipliers”. The SMP is a variational inequality formulated by means of 
the state trajectory and the dual process. It is satisfied by any optimal control and, usually, adding 
some convexity assumptions, it fully characterizes the optimality. Moreover, if the control enters 
the diffusion, the irregularity in time of the noise forces to introduce a second adjoint process which 
is strictly related to the Lyapunov equation for the first variation of the state. 

The first general formulation of the SMP is due to Peng 17| for finite dimensional systems. 
After this seminal work, there has been a large number of works on this subject, both in finite 
and infinite dimensions for different formulation of the control problem. SMP in inhnite dimension 
has been studied e.g. in Tang and Li [ 2 ^, Fuhrman, Hu and Tessitore [^, Du and Meng [^, Lii 
and Zhang whereas some of the results in finite dimension comprise: Jump control: Tang and 


Li 


21], 0ksendal and Sulem 11 j; Singular control: Bahlali and Mezerdi [H, Dufour and Miller 
[^, 0ksendal and Sulem 12]; Impulse control: Wu and Zha ng 1^ : Delayed controlled systems: 
0ksendal, Sulem and Zhang [1^; Near-optimal control: Zhou [2^ and many others. 


This paper is a natural continuation of 0 on one side and [I^ on the other. Our aim is 
to control the behaviour of a dissipative system in an unbounded time interval and to provide 
necessary conditions for optimality. If Wt is a d-dimensional Brownian motion, the equation for 
the state can be written in the form 

Xt = x+ / h{Xs,Us)ds+ / a{Xs,Us)dWs, 

Jo Jo 

P—a.s. for all t € [0, -|-oo) and all x € M"'. The objective is to minimize a discounted functional 

r+oo 

J{ 


T+00 

(u(-))=IE/ e-^^fiXt,ut)dt, 
Jo 
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over all admissible controls u{-) with values in a general separable metric space {U,d). Let us remark 
that this general assumption on the space of control actions allows us to consider a broad class of 
controls, such as bang-bang controls, which are excluded from the classic convexity framework. On 
the other hand, it necessarily forces us to formulate the SMP via a spike perturbation argument. 

The analysis of the problem is based on the well-posedness of the state equation under a joint 
(or global) monotonicity assumption. For any G and fixed p > 0, there exists Cp G M such 
that 

{b{x,u)-b{y,u),x-y)+p\\a{x,u) - a{y,u)\\l < Cp\x - y\^, e U, (1) 

where U is the space of control actions. For a detailed exposition of SDEs with this property see 
[^. This condition is a generalization of the usual dissipativity condition which involves only the 
drift of the equation and allows us to consider a larger class of concrete examples. Informally, there 
is a balance between the dissipativity of the drift and the noise term. If the drift term is dissipative 
enough, the diffusion term can grow in a polynomial way, instead of being globally Lipschitz. As 
a reference for this general assumption, see and (I^ . A list of several important examples 
satisfying joint monotonicity condition is given further in this paper. However, let us notice that 
many interesting equations do not satisfy a global monotonicity assumption (see [3] for a selection 
of examples) and the formulation of a version of the SMP for these systems could be a subject of 
a future work. 


In the first step of our analysis we have to deal with a partially-coupled system of the state 
equation and the first adjoint equation. The delicate question here consists in giving a precise 
meaning to the solution of the following backward SDE 


dpt 


D^H{Xt,ut,pt,qt) 


rpt 


dt qtdWt, 


( 2 ) 


where H{x,u,p,q) = {p,b{x,u)) -|- Tr [q'^a{x,u)] — f{x,u) is the Hamiltonian of the system and 
[X,u) is an optimal pair. 

In general, the behaviour at infinity of BSDEs is not easy to understand and different approaches 
and approximations are proposed in several papers. In our setting, we are able to tackle the problem 
showing that the adjoint equation preserves, in some sense, the dissipativity of the state. Using 
a duality argument and the same technique as in (l^ we can show that there exists a solution 
in some exponentially weighted space. It turns out that for the analysis of the state and adjoint 
equations the condition on the discount factor (forming both the functional and the weight 
in the definition of exponential weighted space) is given by some formula in terms of the joint 
monotonicity constant Cp G M. Nevertheless, due to the framework of SMP (use of the spike 
variation techniques) and, more importantly, due to the form of the polynomial growth assumption 
one has to assume implicitly r positive so that the polynomial bound is integrable with the weight. 


As already mentioned, the presence of the control in the diffusion term makes a second adjoint 
process to appear. In this case, there exists a formal matrix-valued BSDE which represents the 
process, but due to the lack of dissipativity of the equation, it seems to be impossible to obtain an 
a priori estimate of the solution. To solve the problem we follow the idea of Fuhrman et al. in 
and we define the first component of the second adjoint process Pt as a bilinear form defined via 
the relation 

/ OO 

{DlH{Xs,Us,Ps,qs)yi'^yl’'^) ds, (3) 

where {yi’^)s>t is the solution of the linearized state equation starting from r] at time t. Note that 
the second component Qt does not appear in the definition of the SMP. Proceeding this way it is 
not necessary to define and solve the second adjoint equation (i.e. finding the couple {P,Q)) but 
it is sufficient to identify only the process P via the equality Notice that our definition of P is 
similar to the notion of transposition solution presented in [9(]. Nevertheless, if the diffusion term 
a is Lipschitz, P can be indeed identified as a unique solution to a matrix-valued BSDE which, 
in fact, inherits the monotonicity property from the state equation. The formulation of the SMP, 
in this case, follows by similar arguments as in (I^ but with an extension to the infinite horizon 
setting. 

Having in mind the form of the Hamiltonian of the system, the final step (and the main result, 
Theorem [S]) is to derive a necessary condition for optimality. Let us suppose that {X,u) is an 
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optimal pair, then for every v U, the following variational inequality has to hold P (8) dt—a.e. 

1 

H{Xt,v,pt,qt) - H{Xt,ut,pt,qt) + 2 - a^{Xt,ut)) ,a^{Xt,v) - a^{Xt,ut)'^ < 0. 

i=i 

This variational inequality can be rewritten in terms of so called ?^-function defined by 
n{x,u) := H{x,u,pt,qt) - ^TT{a{Xt,ut)'^Pta{Xt,ut)) 

+ ]^Tv[{a{x,u) - a{Xt,ut))'^Pt{a{x,u) -a{Xt,ut))], 

in the following manner 


T-L (Xt,ut) = raaxP (Xt,v) , F 0 dt — a.e. 

^ vGU ^ ^ 

The paper is organized as follows. The second and third sections contain the basic assumptions, 
the formulation of the discounted problem and a list of motivating examples. In section 4, we study 
well posedness of the state equation. The fifth section concerns with the application of the spike 
variation technique to our problem. In the two next sections, 6 and 7, the two adjoint processes 
are studied. The first adjoint BSDE is solved by approximation and a duality argument whereas, 
the construction of the process P is described without any potential relation to some BSDE. A 
precise statement of the main theorem (Theorem [5]) is presented in section 8 and its proof is given. 
Last, in the Appendix we provide some technical proofs and we quote the actual restriction for the 
discount factor including the final one used in formulation of the main result. 


2. Assumptions and preliminaries 

Let W = {IE/,...,IE/},>o be a standard d-dimensional Brownian motion defined on some 
complete filtered probability space (Pt)t>o filtration (Pt)t>o assumed to be the 

canonical filtration of W completed by P—null sets. The space of control actions is a general metric 
space P endowed with its Borel fi-algebra B{U). The class of admissible controls is defined as 
follows 

U := {rt(') : M+ x D —>■ 1/ : u(') is {Pt)t>o ~ progressive}. 

For r € M, p > 1 and a Banach space E, we define 




]E) :=|u(') : M+ x D ^ E : u(-) is {Pt)t>o ~ progressive 


and E 


j 6 < °°}- 


We want to study an infinite horizon stochastic control problem in M” of the form 


dXt = b{Xt, ut)dt + a{Xt,ut)dWt, 
Xq = X, 


Vt > 0, 


( 4 ) 


( 5 ) 


where x G M and tt(-) is an admissible control. The discounted functional to be minimized is given 
by 


|■+oc, 

J(n(-))=E/ e-^^fiXt,ut)dt. (6) 

Jo 

By I • I we denote the Euclidean norm on M”, H-H stands for a Frobenius norm on and, finally, 
11-112 denotes the Hilbert-Schmidt norm on By 5” we denote the set of symmetric matrixes 

]^nxn_ XA denotes the characteristic function of a set A. 

Hypotheses: 

(HI) (U, d) is a separable metric space. 
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(H2) (Polynomial growth) The vector field b : M"" xU ^ M”" is 0 ;S(f7)-measurable and 

the map x e->• b{x,u) is of class C^. Moreover, there exists m > 0 such that 


sup sup 
ueu xsK" 


\Dxb{x,u)\ 
1 + 


< + 00 , 


|/ 3 | = 0 , 1 , 2 . 


(7) 


(H3) (Polynomial growth) The mapping a : M"" x 17 —>■ is measurable with respect to 

;B(M”) 0 I3{U). Moreover the map x i—> a{t,x,u) is C^(M”;M”’^'^) and there exists m (same 
as in (H2)) such that 


sup sup 
u£U a;gR" 



< + 00 , 


|/3| =0,1,2. 


( 8 ) 


(H4) (Joint monotonicity) Let p > 0. Then there exists Cp € M such that, 

{Dxb{x, u)y, y) + p||H^cr(x, n)y||2 < Cp\y\^, x,y e M'", u £ U. 


(9) 


(H5) (Cost) The function / : xU —>■ E is 0(E'^) 0;S(17)-measurable and the map x i-A f{x, u) 

is of class C^. Moreover, there exists I > 0 such that 

sup sup -— ^ < + 00 , |/3| = 0,1, 2. (10) 

u£U xgR" 1 + \x\ 

Remark 1. 

(a) In 1^, the form of (H2) and (H3) is given in a stronger way. For our purposes the above 
formulation is sufficient. 

(b) It is possible to show that condition ([2D implies that for any x,y € E"’ 

{b{x,u)-b{y,u),x-y)+p\\a{x,u) - a{y,u)\\l < Cp\x -yf, u£U. (11) 

and the two conditions are equivalent for coefficients twice differentiable (in x) which, in 
fact, is our case. 

(c) The Mint monotonicity condition m also implies the so called coercivity condition (see 
i.e. [l9|, page 44). Indeed, let us fix y = 0, then (fTTD reduces to 

{b{x,u) — b{0,u),x) + p\\a{x,u) — a{ 0 ,u )\\2 < Cp|x|^, (12) 

and 

2 I 12 

I|cr(x,u) - o-( 0 ,u )||2 > |||(T(x,u )||2 - ||(T( 0 ,u)|| 2 | 

= ||o-(x,u)||2 + ||o-(0,u)||2 - 2||a(x,u)||2||o-(0,u)||2 
> ||a(x,u)||2 + ||o-(0,u)||2 -e||o-(x,n)||2 - 

= (1 - e)||cr(x,u )||2 + ||cr(0,u)||2 - Ve € (0,1). 

Then after easy computation we obtain that 

(6(x,u),x)+p(l - e)||(T(x,u)||2 < .^p(l + |x|^), e€(0,1). (13) 

Let us note that |6(0,u)| + |(t( 0,n)| < C due to the polynomial growth (H2)-(H3), hence 
Kp can be chosen as Kp = max{cp + 1/2, C'^ j2, 1}. 

(d) The above Hypotheses (H2)-(H5) can be generalized to the situation of time dependent 
stochastic coefficients under natural assumptions without any influence on the main result. 
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3. Motivations and examples 


Apart from the classical Lipschitz setting, there are two usual sets of assumptions which assure 
global existence and uniqueness of the solution to an SDE. The first one consists in local Lipschitz 
property of the coefficients along with the so called coercivity condition 

{b{x),x) + ^||o-(x )||2 < K{1 + \xf), (14) 

for all X G M”" and some iL G M. The second set comprises some dissipativity assumptions on the 
drift, still with Lipschitz diffusion term. The dissipativity is expressed by 

{b{x) - b{y),x -y) < K\x - ^/|^ 

for all X, y G M” and some AT G M. 

Another step further in this direction is the so called global monotonicity assumption, which is 
formulated as a joint condition for drift and diffusion 

{b{x) - b{y),x- y) + ^||cr(x) - a{y)\\l < c\x - yp, (15) 

for all x,y G M”". It is important to mention that the joint monotonicity property immediately 
implies the dissipativity of the drift but not necessarily global Lipschitzianity of the diffusion part. 
Moreover, the joint monotonicity (|15l) also implies the coercivity property (jl4p . 

It turns out that for the purposes of the SMP it is natural to strengthen the global monotonicity 
assumption in the following form: for all fixed p > 0 there exists Cp G M such that 

{b{x) - b{y),x- y) +p||cr(x) - u(y )||2 < Cp\x - yp, (16) 

for all X, y G M"’. This is motivated by the attempt to solve not only the state equation, but also the 
first and second variation equations and to derive some appropriate estimates of higher moments 
of the solutions. Another natural assumption in this framework is the polynomial growth of the 
coefficients along with their derivatives. This is fundamental in order to choose the correct discount 
factor in the definition of the weighted spaces M”) that we are going to use. Let us also 

mention that these polynomial bounds allow us to prove the local Lipschitzianity of the coefficients 
of the state equation. 

To conclude, notice that the freedom in choosing p in the definition of (I16p implies the existence 
of a link between the growth of the diffusion term and the drift (compare (H2) and (H3), see also 
0). For example, to gain a quadratic growth in the diffusion we have to require the system to be 
more dissipative. Concretely, one such an example is 

dXt = [Xt - Xl] dt + XfdWt. (17) 

A more general framework is presented in (I^ and @| where the authors do not ask for polynomial 
growth of the coefficients and present a weak local version of the global monotonicity assumption 
along with a weak coercivity assumption. By weak we mean the presence of stochastic coefficients 
instead of constant ones in the definitions of the conditions, see [l9l] for a detailed exposure. 

After this preliminary discussion we also present some concrete models. 


(1) Polynomial model: As we have discussed above, a one dimensional model given by the 
SDE 

2m 


dXt = 




2=1 


dt 




. 2=1 


dWt: 


Xo = X G 


for some ai,bi G M, is the simplest example we have in mind. Let us notice that we can 
easily generalize the model in a way so that these polynomials are upper bounds for some 
more general (but locally Lipschitz) functions satisfying the joint monotonicity condition. 


(2) Population growth models: A model in M given by the SDE 


dXt = aXth (Xt) dt + aXtdWt; Xq = x > 0, 
where h{x) = 1 — /31n(x), for so called Gompertz growth models (tumor growth models 
etc.) or h{x) = 1 — /3x, for so called logistic growth models (population dynamics models 
etc.). A detailed discussion of the controlled logistic model on infinite time horizon can 
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be found [10|]. In both cases, a > 0 is the speed of growth and /3 > 0 represents some 


saturation level of the system. It can be shown by Lyapunov techniques that the solution 
is positive and an explicitly analytic formula can be found by linearizing the two equations. 
It is important to mention that our version of SMP covers the case of controlled logistic 
models (in full generality) whereas the controlled Gompertz model can be treated only with 
uncontrolled diffusion. This fact is due to the lack of polynomial growth condition needed in 
(H2) and the second variation equation might not be well posed. The same argument holds 
for another generalizations of the two population models with different choices of diffusion 
term (crx(l — ln(x)), iTy^x(1 — \n.{x)), ax{l — x) etc.). 


(3) Gradient flow model with stochastic perturbation: Let K he a. convex open subset 
of and (^ : iL C —>■ M a A-convex function, i.e. 

A 2 

ip{{l - a)xo + a^i) < (1 - a)(/?(xo) + cinp{xi) - -a{a - l)|a:i - xo| , 

for every xq,xi € and a € [0,1]. Then we can study a SDE of the form 

dXt = -V^{Xt)dt + a{Xt)dWt, 

provided that (/? is of class C^. In fact, A-convexity (with C^-regularity) is equivalent to 

(V(/j(?/), y -x) -^\y-x\^ > ip{y) - ip{x) > {Vip{x),y - x) + ^\y - xp 

which in particular implies that Xtp is A-dissipative. If we ask a to be Lipschitz, then (llip 
is satisfied. 

Some possible choices of ip{-) are: 

• Take A = 0 and (p{x) = \x\^^ convex with the derivative 2k\x^^~‘^x] 

• (Double-well potential) Let d = 1 and consider ^p{x) = {x^ — 1)^, which is not convex 
(±1 are minima) but A-convex. 

• Let d = 2 and consider the following dynamics 

f dXt = -Xtdt + XtY^{l + X^)-^dt + adwi^\ 

[dYt = -Ytil + X^)-^dt + adwi^l 

Here the energy has the following form ^ + (p 2 : where 

x‘^ 

^i{x,y) = — and yP 2 {x,y) = 2(^1 ^ 

The difference between this case and the previous one is that here, the energy has not 
isolated minima but rather forms a sub-manifold (i.e. the x-axis). 


4. State equation 

In this section we provide the existence and uniqueness theorem for the state equation m- 
The classical proof of such theorem under our Hypotheses (H1)-(H4) goes along the lines as in 
[^, Section 1.2. for a solution in the space ([0, T]; M"") (thus not in exponentially weighted 
space ([0, T];M"")). Nevertheless, by these arguments one can not obtain a contraction from 

([0, -|-oo); M") to itself (not even locally in time). Hence an another approach has to be chosen. 
More specifically, we will scale the original equation in a way so that the classical result from 
can be applied. We stress that, in this case, the bound for the discount factor depends on Cp which 
can be also negative. 

Theorem 1. Let Hypotheses (H1)-(H4) hold. Then for every admissible control u{-) there is a 
unique solution process {Xt)t>o to SDE with supjg[o,r] < + 00 , for each T > 0 and 

for r > 2ci/2- Moreover, for all q >2 and for r > 2cq_i it holds 

poo 

E / for some constant Gi = Ci{q) > 0, 

Jo 


(18) 
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where Cq-i is the joint monotonicity constant in ( 0 j. In addition, for q ^ and r > 4ci, it 

holds 

roo 

E / e~'^^\Xt\‘^'^dt < C 2 \x\^, for some constant C 2 = C 2 {q) > 0. (19) 

Jo 

Proof. Without loss of generality we prove the result for uncontrolled case (which can be easily 
converted to the controlled one). The idea is to apply the result from to a transformed equation 
which corresponds to our original equation. Assume for a moment that we already have a process 
X satisfying the dynamics given by ([5]) and define Xt = e~^^Xt. Then X solves 



Denoting b(t,x) = —+ e“ 2 * 6 (e 2 *x) and a{t,x) = e“ 2 *cj(e 2 *x) it is easy to check that b,a 
also satisfy assumptions (H1)-(H4). Differentiability and polynomial growth (H2)-(H3) are evident 
whereas joint monotonicity (H4) holds in the following sense 

{b{t,x)-b{t,y),x - y)+p\\d{t,x) - d{t,y)\\l < Cp\x-y\‘^, Cp = Cp-^. (21) 

Hence, due to [^, Section 1.2 there exists a unique predictable process X solving (f20|) which satisfies 
sup^g[Q 2 ^] E|Wp < + 00 , for each T > 0. But this means that there actually exists a process X 
solving dS]) with the following integrability property 

sup E[e"'’‘|At|2] = sup E|At|2 <+ 00 , VT > 0. (22) 

te[o,r] te[o,T] 

The last step is proving the desired exponentially weighted integrability. Denote the exit time 
tk ■= inf{t > 0 : \Xt\ > AT} for each AT > 0 with the usual convention inf{0} := + 00 . It is easy to 
show that Tk T = + 00 , for AT —>■ +00 a.s. 


Proof of estimate m- Let us fix g > 2 and apply the ltd formula to {XtATK^'^- We also de¬ 
note ds = d{s, Xs)d{s, Xg)'^■ Then we obtain 

nXtArj.f'^ = + 2qE 6 (s, X,)) + ^Tr ds 


+ 


2q{q - 1)E X{r^>s}\Xs\‘'^^-^^Tr {a, (x, ® X,) } ds 


< -b 2gXg_iCg_iE J X{tk>s} + l) ds 

= \x\'^'‘+ 2qKq_iCq-i{t Atk)+ 2qKq_iCq-i [ ElXsATK^'^ds, (23) 

40 

where we have used the joint monotonicity in the form of (ffH) and coercivity-type estimate (fT^ . 
By Gronwall lemma it easily follows that 

ElXtATK^'^ < (\x\‘^‘> + 2qKq_iCq-l{t ATk)^ < |^|2gg2gK,_i (c,_i-§ )t _ 

The last estimate can be made for r > 2cq-i. Consequently, the final estimate reads 

E|Xt|^'' < |a;|2ge2gA:,_i(c,_i-0t^ 

and it follows by Fatou. Expressed in terms of the process X we have that for all t > 0 it holds 

< |2;|2'?g2QA'g_i(cq_i —r)4^ 


E 


g-rtg|x^|2g 


( 24 ) 


Now it is sufficient to integrate both sides of (IMl) on [0, +oo). 
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Proof of estimate (fTUD : Fix q>2 and observe that . Therefore, applying 

Ito formula to gives 

rt 


E 




\2q 


= IxP'' 


+ 2 #/ 
Jo 


1 , 


X ( {Xs,b{s,Xs)) + -Tr{ds}]ds 


2q{q-l)E (x, ® X,) } 


+ : 

+ r(g-l)E 

Jo 

< \x\^^ + 2qKq_iCq.iE 1'^ {jXsf'^ + l) ds 


ds 


iq-l)E rx{,^>,}e^(«-i)^|X,|2'?ds 
Jo 


+ r[q - 

< \x\^’i + 2qKq_Xcq-i{t A tk) + (2qKq_Xcq-i + r{q - 1)) |X,Arx 

Then, similarly as before, we obtain 

poo 

E e-^'lXip-Jdt <(721x125, for r > 2(7iFg_iCg_i, (25) 

Jo 

To conclude the proof observe that once we have obtained the estimates (1251) for g > 2, the case 
q G [1/2,2) easily follows by Holder inequality. Note that we have proved even more than stated 
in the assertion of the theorem. Nevertheless, such generality is not needed for the purposes of 
proving the SMP. □ 


5. Spike variation and variation equations 

The derivation of the variational inequality needed for the formulation of SMP is based on 
expanding the difference of the functional J {u^{-)) — J {u{-)) where u{-) is an optimal control and 
u^{-) is its appropriate perturbation. Since the control variable is allowed to enter also the diffusion 
term, the expansion has to be carried out up to second order due to the time irregularity of the noise. 
Therefore, two forward variation equations appear in our setting: first order variaton process y‘^ 
being, in fact, a linearization of the state process, and the second order variation process coming 
from the second order expansion. We also stress that due to the estimation techniques used in the 
forthcoming proofs, we often need the polynomial bound of coefficients to be integrable with the 
weight, which immediately implies that r has to be a priori positive. 


Let s > 0, C M+ be a set of measure e of the form := [to,io + ^]) with tQ > 0 arbitrary but 
fixed, and tt(-) an optimal control. Define the spike variation n^(-) of u{-) by the formula 


u 


€ 

t 


ut, at €R+\Es, 

V, if t € Ee, 


where u € [/ is an arbitrary and fixed point. 

Let (X(■),'«(•)) be a given optimal pair and (X^(-), u^(-)) satisfy the following perturbed SDE 


dXf = b{Xf,ul)dt + a{Xf,ul)dWt, 

XI = X. 


(26) 


Further, following the notation of Yong and Zhou [2^, we denote by 6ipt the quantity ip{Xt, 
rtf) — ip{Xt,ut), for a generic function ip. 


Now, let us begin studying the first variation equation 

f dyl = D^b{Xt,ut)ytdt + [Dxcr^ (X*, ut)yl + dal] dW^ , 

Ug = o, 

and the second variation equation 


( 27 ) 
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dzt = 


+y:U 

where we have adopted the notation 


Uo = 0, 


D^b{Xt,ut)zf + SbtXEeit) + ]^Dlb{Xt,ut){ylf dt 

^D^a^{Xt,ut)zf + 5{D^al)ylxEe{t) + {Xt,ut){yff 


dW}, 


(28) 


Dlb{Xt,utWtf := 


/ T,[Dlb\Xt,ut)ym)^] 
V Tv[Dlb^{Xt,ut)yt{ytf] 


/ T,[Dla^^{Xt,ut)yt{ytY] 

Dla\Xt,ut){ylf ■.= \ \ 

\ Tv[Dla^\Xt,ut)ymf] 

Theorem 2. Let Hypotheses (H1)-(H4) hold. Then there exist ri € M, r 2 > 0 such that equation 
(1271) (equation (1281) . resp.) admits a unique solution y^ E ( z^ £ 

resp.) for all admissible controls u{-) E U. 

Proof. Note first that (I27p and (|28p are linear equations in y^ and z^, respectively. The perturbation 
of the diffusion in (1271) belongs to M”) for every r, and it is independent on y^. Therefore, 

the joint monotonicity condition (llip holds and the proof follows by similar arguments as the proof 
of Theorem [H for ri > 2ci/2- Concerning the equation for z^, we have to choose r 2 such that 
D‘^b{Xt,ut){yf)‘^, D'^a^{Xt,ut){yf)'^ E M*^). Then existence and uniqueness of a solution 

follow. □ 



Remark 2. Let us note that, thanks to the linearity of the equation and due to the structure of 
the forcing term dcr^, the solution y^ to the equation (|27p is identically zero for times t < to- 

In the rest of this section, we will often benefit from a general estimate of the solution to a linearized 
SDE given by the following Lemma. 

Lemma 1. Let Y E L‘^~^ (M_|_;]R") be a solution to the following linear SDE 

r dYt = {AtYt + at)dt + ZU i^tYt + Pi)dWi 

I 7b = yo, 


where yo E M"; A,B^ : M_|_ x fl —>■ a,fd^ : M+ x 17 —> M", j = l,...,d, all are (77) — 

progressively measurable processes. Let k > 1, p > 0 and Cp E M such that 


1. {AtYt,Yt) +pEU ^ 


Then it holds 


(g) dt—a.e. 
1 _ 
k 


dt < Too, 1 < j < d, and r > 2c2k-i- 


sup e-''^%Yt\^^ < K 

iGR+ 


nyo\^^ + 


a °° 1 \ 2 fc 


where K = K(6), for some appropriate 6 > 0 and r > 2c2k-i- 



Proof. The proof will be given for all ’s bounded. Then the stochastic integral in the proof is 

a true (centered) martingale. The proof for the unbounded case follows immediately by standard 
localization and the Fatou lemma. 
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Let 2 k, k >2 and apply the Ito formula to e on [ 0 , t]. The case k E [ 1 / 2 , 2 ) follows easily 

by the Holder inequality. 

=E|yoP^ + 2A:E T (H.T, + T,) ds 


2fc-4 / Tjj 


{BiY, + l 3 i,Ys) ds 


d 

+ 2k{k - 1 ) Ve / e-^’^^lYs 
1=1 -^0 

+ Ve r Iy, 12^-2 (H/y, + /?/, H/n + /S^') ds 
1=1 -^0 


r-l 


-rfeE / e-^^"|yj2^ds 


( 30 ) 


—rAis| 2 fc —2 


<E|yoP^ + 2 fcE [ e“’''^"|y, 
Jo 


(H,y„n) + ( 2 fc-l) 

1=1 

^ V 

<c 2 fc-i|yp 

t 


ds 


+ 2 kE I e-'"'^^\Ys\‘^’^-^\as\ds + 2 k{ 2 k-l)E I 


Ids 


-rkE / e“^^"|ysp*^ds. 

Jo 


0 


P rjj LQ 

Now, using Holder and the weighted Young inequality ab < + a, 6 > 0 , <5 > 0 , the remaining 

terms can be treated as follows 


1=1 


i-t / / \ -^N roo 1 

2 kE < 2 fcf sup e"'’*^*E "" / e" 5 *(E|atp^)^ 

do '^iGR+ d / do 

°° r 1 \2fc 

g-r«tE|y^|^« ) + ^( / e- 2 *(E|atp^)^ 

^tGK+ 




2k / \ 1 / 1 

< ( 2 A:-l)d^( sup e-^^*E|yt| 2 M + ^( / e-i*(E|atp^)^dt 

'^tGK+ ^ w ^Jo 

and similarly 

pi ^ 

2 k{ 2 k - 1 )E / \Pi\‘^)ds < 2 {k - l){ 2 k - 1 ) 6 ^ ( sup e-^^*E|yi| 

Jo 2 ^teK+ 

+ 2 { 2 k -l)^( r E P") ' • 


( 31 ) 


( 32 ) 


1=1 


The estimate (| 2 ^ easily follows by substituting (ISTTl and (l3^ into ([30]) . by taking supj>o on both 

2k 2ir 

sides and finally by choosing d > 0 such that 1 — {2k — l)d 2 fc-i — 2{k — l){2k — 1)6^*^-^ > 0 and 
r > 2c2k-i- □ 

Before proceeding, let us recall that by X and we mean the solution to (0) in the space 
(M+; M"'), for r > 2 ci/ 2 ! corresponding to u(-) and u^{-), respectively, and are the 
solutions to ([271) and ([28l) . respectively. 

Proposition 1 . Suppose Hypotheses (H1)-(H4) hold and r > 2 ci/ 2 - Define := Y/ — Xt, yf := 
~ Ut Ct ■— Ct ~ Vt ~ i > 0. Then there exist pi, ..., ps > 0 sueh that for k = 1,2,... it 
holds 

(i) supieR^e-^i"^E|^f| 2 " = 0 (e"), 

(ii) sup,eR^ = 0 (e"), 

(hi) SUPi^K^ e-^3HiE|^e|2fc ^ 

(iv) sup,eR^ e-^ 4 "*E| 7 ?f = 0 (e 2 fc), 

(v) supieR e-P^^^E\Cff’^ = 0(6^^). 
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Proof. See Appendix. 

Before giving a preliminary expansion of the cost, we state the following 
Lemma 2 . If g £ then the following equality holds for every x,x £ M” 

g{x) = g{x) + {Dxg{x),x — x) + f (6D‘^g{9x + {1 — 9){x — x)),x — x) d9. 

Jo 

Proposition 2. The following expansion holds for the cost functional 

poo 

J{u%-)) - J{u{-)) = E / + zf) dt 


□ 


+ E 


/ 


oo r 1 


§ {DlfiXt, ut)yt, yl) + df{Xt, ut) 


(33) 


dt + o(e), 


where the discount factor r > maxj=i^..,^5{/9j} and pi are the individual discount factors from Propo¬ 
sition [il 

Proof of Proposition Thanks to Lemma [21 we have 

/ OO 

e-'-* [f{Xt,ul)-f{Xt,ut)]dt 


= E 


[5f{Xt,ut) + {D^f{Xt,uf),et)] dt (34) 

JO 

+ E ^ {9Dlf{9Xt + (1 - e)Xf, 


)d9dt. 


Finally, Proposition [T] gives 

J(ir^(-))-J(h(.))=E 




/O 


5f{XuUt) + {5D,f{Xt,ut),et) 


+ {Dxf{Xt,ut),yt + ^t) + {Dxf{Xt,ut),CD 
+ 1^ (9 {Dlf{9Xt + (1 - 0)Xf,ut) - Dlf{Xt,ut)) ilii)d9 
+ i {5Dlf{Xt,ut)fiet) + ^ {Dlf{Xt,ut)y!,yt) 


+ ^ {Dlf{Xt, ut)gl, ft + yt] 


dt 


= E 


e {D^f{Xt, ut),yt + zt) dt 


+ E 


f 




o i^lfiXt, Ut)yt,yt) + 6f{Xt,ut) 


dt + o{e), 


which completes the proof. 


□ 


6. First adjoint equation 


The first adjoint process naturally arises as a solution to an appropriate BSDE whose driver can 
be obtained by differentiating the Hamiltonian function with respect to the state variable x. In 
some sense, the first adjoint process is dual to the linearized state equation (l27|) and it can have the 
interpretation of generalized (in the sense of time-dependent and stochastic) Lagrange multipliers. 
In the classical setting for BSDEs, the terminal condition is given a priori. On the contrary, here 
the BSDE is solved on infinite time horizon and the behaviour at infinity is not known. Yet the 
existence and uniqueness result can be derived for processes being in some exponentially-weighted 
space. 

In our case, the first order adjoint equation on infinite time horizon has the following form 


dpt 


D^b{Xt, utfpt Dxa{Xt,utf qt - D^f{Xt,ut) 


rpt 


dt -\- qtdWt, 


(35) 
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where Dxa{Xt,ut)^qt := ql € M” and {Xt,ut) is an optimal pair. 

Let us start the analysis by proving an a priori estimate for the difference of solutions to (|35p . To 
do so, the following estimate will be of a particular interest since it allows to transfer the joint 
monotonicity property to the BSDE 

d d 

{D^a{Xt,utf quPt) = ^ {D^a^{Xt,utf qi,pt) = ^ {q{,D^a^{Xt,ut)pt) 

i=i i=i 




(36) 


dl^. 


i=i 


i=i 


'2,-r / 


Lemma 3. Let (p^, q^) and {p^, q^) he two solutions to BSDE (1351) belonging to the space Ljr ; 


pnxd 


), for r > 2 ci/2 ; corresponding to f = and f = /^. Then the following estimate holds true 


E 


f 


^—rt 


(r - 2 ci /2 - S)\pl-p‘ff + -\\ql - ql 


2||2 


<iE 


dt 


e-^^\DJ\Xt, ut) - DJ\Xt, ut)\^dt 


where <5 > 0 is sufficiently small. 

Proof. Applying Ito formula to — Pt\^ gives 

ipj - +E r^-"' (---iri -pd+11?; -9fit) 


dt 


f 


f 


= E 
+ E 

< E 
+ E 
+ E 

< E 


2e (pI - pf, D^b{Xt, UtYfp^ - pf) + D^a{Xt, UtY {q^ - qf)) dt 


\T/1 


\T/„1 


f 

■oo 

/ 

f 


2e {pj - pi, -r{pi - pi) + D^fYXt, ut) - D^f{Xt,ut)) dt 


—rt 


{pi-pi,DxbiXt,ut)'^ipl-pi)) + f-\\D^a{Xt,ut){pi - pi)\\2 ] dt 


1 , 


2 m |2 




2e 


—rt 


\pl-pi\" + l^\DxfYXt,ut) - DJYXt,ut)\" ) dt 


. 2|2 


1 


2 h 


f 




1 , 


( 2 ci /2 -2r + 5) \pI - pif + -\\ql - qj 


2||2 


+ E 


fOO 1 


2 ' 

- m 2 


dt 


e --\D,fYXt,ut) - D,fYXt,ut)rdt, 


where we have used the estimate (13611 . joint monotonicity assumption (H4) and weighted Young 
inequality. The conclusion easily follows. □ 

Before giving the proof of existence and uniqueness for the first adjoint equation we produce a 
preliminary result in finite time horizon T > 0. Let us consider the following equation: 


\dpt = - Dxb{Xt,ut)^pt + D^a{Xt,ut)^qt - D^f{Xt, ut) - rpt dt + qtdWt 
[PT = 0, 


(37) 


where T > 0 is arbitrary but fixed. 

As far as we know, no results in the literature can be used to solve this equation due to the 
polynomial growth of DxCr{Xt,Ut)'^ in front of qt- In order to produce existence of a solution to 
such equation we exploit some duality arguments. 

Theorem 3. Under Hypotheses (H1)-(H5) equation (l37|) admits a unique solution {p,q) which 
belongs to G ^^.([O,T];M*") xL3.([0,T]; for each T > 0. 
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Proof. The proof consists in three steps. First, the diffusion term a is approximated so that 
there exists a solution for each approximating backward equation, by classical results. Second, a 
duality between these approximate solutions and a properly perturbed first variation equation is 
established. The last step consists in constructing a unique solution to the original equation (on a 
finite horizon) by some compactness arguments. 

Let us define a sequence of Lipschitz-continuous maps cr"' with a^{x) a{x) as n —oo, for all 
X € M" so that the joint monotonicity property still holds. An example of such approximation can 
be given by (see i) 


a-ix) = 


a{x), 

(n+l)fc 


a 


Then, for each n, the following approximating equation 


if \x\ < n, 
if |x| > n + 1 . 


dp^ = - [D,b{Xt, UtY p^ + D^a^{Xt, UtY - DJ{Xt, Ut) - rp2\ dt + q^dWt 

[p^ = 0 , 


(38) 


admits a unique solution (p^qt) ^ ^ L^([0, T]; thanks to Briand et al. [ 2 ], 

Theorem 4.1. 


Now, to establish the duality, consider for all n € N and for all 7 (-) € L^([0,T];M"") and ?/(•) € 
L^([0, T]; the following perturbed first variation equation 

f dpt = {D^b{Xt,ut)yt - ryf + 74 ) dt + ut)y? + Vt) dWt, t G (0, T], 

[Vo =0. ^ ^ 

Due to Theorem [2] we know that the above equation has a unique solution in L^([0, T]; M"') for 
each n. Moreover, using dissipativity it is easy to show that there exists K > 0 not depending on 
n such that 


E 


Tly; 

Jo 


?\‘^dt < K 


E / | 7 t|^dt + E / WptYdt 


(40) 


Next, by applying the Ito formula to d {yf^p't) we establish the duality relation 


E 


£ (pY 7 t) dt + ^£ Tr [qf {ptf] dt = -E £ {D,f{Xt, ut),y^) dt. (41) 

Let us define the set A := { 7 (-) € L^([ 0 , T];M"") : || 7 ||j ;^2 ([orj-K'*) — r/ = 0 in ()4T]l we 

get 


1/2 


E/ \p^\^dt] < 


sup 


/ 7eA L 40 


< sup 

-y&A 


E/ \{D,fiXt,ut),y?)\dt 


\ V2 / i-T 


1/2 


< 


^ \D^f{Xt,ut)fdt] (E 

c(EjJ\D^f{Xt,ut)fdt^ 


[ \y\ 

Jo 


n\2 


dt 


1 / 2 - 


(42) 


If we repeat the same argument with 7 = 0, instead of r/, we finally get 

rT 

ll4’rilL^([0,r];K'*) I \dJxf Ut)\ dt. 

J 0 

This way we have obtained a uniform estimate (with respect to n) of the L^—norm of {p^,q'^). 
Hence there exists a subsequence, denoted by abuse of notation again as (p"", <?"■), which converges 
weakly in L^([0,T];M"’) x L^([0,T];M"’^'^) to a couple {p,q). Our goal is to verify that {p,q) is the 
solution to the limit equation 


\dpt = - Da;b{Xt,ut)'^pt + Dxa{Xt,utYqt - D^f{Xt,ut) - rpt dt + qtdWt, 

\pT = 0 . 
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To do SO, we note that due to the linearity of the equation, it is enough to prove that each term 
of the approximating equation weakly converges to the corresponding term in the limit equation. 
Let us start by studying the term 


{D,a^iXt,Utfq?,v) = {q^, D,a^{Xt,Ut)v) 

= {qt,D^a{Xt,ut)v) + {q^,D^a'^{Xt,ut)v - D^a^{Xt,ut)v) . 

As n —)■ oo, the right hand side converges to 

{qt,D^a{Xt,ut)v) = {D^a{Xt,ut)'^qt,v), 

thanks to the pointwise convergence of the derivative of Indeed, D^cr'^ix) = Dxcr{x), if |a:| < n, 
and the derivative is bounded. Regarding the noise term, let us notice that the map q —>• qtdWt 
is linear and continuous, hence weakly continuous. The other terms are easy to treat. 

For the uniqueness part it is enough to use a version of Lemma E] on finite time horizon. Then we 
have existence and uniqueness of a solution in finite time horizon and the proof is finished. □ 

Remark 3. 

(a) The introduction of the term —ryf in (l3^ is due to the choice of the scalar product used 

for establishing duality. If one considers a scalar product in rather than in then 

the additional term —ry” can be omitted. 

(b) An alternative approach to obtain the uniform estimate can be the one following Pardoux 
[I^ . Indeed, applying the Ito formula gives 

EbrP = + {p^,,Dxa^{Xs,utfq^,) + (K, n,))] ds 


-2rE / |p^|^ds-E 


II2 


ds. 


which, thanks to the joint monotonicity of b, cr” and weighted Young inequality, produces 


E|p-|2+ (2r - e - 2ci) E \p^\‘^ds + (^1 - E ^ 


"ds 


< -E 


\Dxf{Xs,Us)\‘^ds, 


for all t G [0,T], e > 0 and r > ci. Again, we have a uniform estimate (in n) for the left 
hand side and the relative compactness argument can be applied as before. Note that this 
approach gives another restriction on r than the one used in the proof. 

(c) All the results of this section can be made more general when considering general weighted 


Young inequality ab < p > 0 i 


with p = 1. 

Now we are ready for the following 


m 


rather than the usual Young inequality 


Theorem 4. Under Hypotheses [HI) — {HU), there exists r > 0 sueh that equation (|3^ admits a 
unique solution {p,q) whieh belongs to x 

Proof. Following Peng and Shi (I^ . Theorem 4, define for all A: G N 

Pt ■= Dxf{Xt,ut)x[o,k]{t), t e ^+, 

which converges to Dxf{Xt,ut) as k ^ oo. We define the solution to the following approximate 
equation on infinite time horizon 

dp’f = - Dxb{Xt,ut)^p^l + Dxa{Xt, ut)^q^ - (ft - rp^ dt + q^dWt, 
as a process solving the following BSDE on a finite time horizon 

f dpt = - Dxb{Xt,ut)'^p^ + Dxa{Xt, ut)'^qt - Pt - '<"Pt dt + qtdWt, 

m = o> 


(43) 


( 44 ) 
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and which is identically zero for t € (fc, oo). Such solution exists for each k due to Theorem [31 
Using the a priori estimate given in Lemma [31 it is easy to see that there exists r such that the 
sequence of solutions {Pt,qt) forms a Cauchy sequence in M"") x and 

that the limiting processes {pt,qt) solves ([35]) . Uniqueness is straightforward due to Lemma [31 □ 

7. Second adjoint 

The second adjoint equation has the following form 


-dP{t) = D,b{Xt, utfPt + PtD,biXt,ut) 


+ ^ (Xt, utfPtD^a^ (X*, ut) 

j=i 

d 

+ Y, (D^a^{Xt,utfQ{ + Q{D^a^{Xt,ut)) 


(45) 




+ DlH{Xt,ut,pt,qt) - nPt 


dt-YQldWi. 


For a detailed discussion of the role of this equation see e.g. [2^. We can see that the term 
DxCrHXt,ut)'^PtD^a^{Xt, ut) destroys the dissipative behaviour of the dynamics in the sense 
that, in general 

d 

i CpWPtWl (46) 

Nontheless, see Remark [H for one particular case. The lack of dissipativity prevents us from finding 
an a priori estimate of the solution. Hence the argument we adopted to solve the first adjoint is 
no longer helpful. The only information that can be useful to study the process Pt comes from the 
first variation equation (12711 . It can be shown that Pt is dual (in some sense explained later) to the 
process Yf defined as Yf = yf(yf)^. It is not difficult to verify that Yf is a symmetric and positive 
(semi)definite matrix process. By using Ito formula it can be also shown that it is a solution to the 
following (matrix-valued) SDE 


where 


dYt^ = 


Dxb{Xt,ut)Yf + YfDxb{Xt,utf 

d 

+ Y Dxa^{Xt,ut)YfDxa^{Xt,utf + r(t) 


i=i 

d 


dt 


+ Y [Dxcr^iXt, ut)Yf + Yt^Dxa^{Xt,utf + A^(t)] dW}, 




= 0, 


(47) 


and 


r(f) := Yd^^i^t,ut) [5a^{Xt,utY + Y,Y>xCr^{Xt,ut)yl {5a^{Xt,ut)Y 


i=i 


i=i 


+ Yd(^^ iY:t,ut){ylY' Dx<T\Xt,utf, 

i=i 


a a 

A{t) = Y^^it) := ^[<5a^(Xi,h0(yD^ + yf {SaHXt,ut)y 


i=i 


i=i 


We also have the following 


(48) 


( 49 ) 
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Proposition 3. Under Hypotheses (H1)-(H5), there exists r G M such that equation (|47l) has a 
unique solution G and the following holds 


/•OO /•OD ^ /*00 

E/ e-^^\\Yf\\ldt < KE e-^^\\Tt\\ldt +K'^E e"^*||A^'(50) 
Jo Jo Jo 

for some K > 0. 

Proof. See Appendix. □ 


The crucial point here is that Proposition [3] holds true if and only if Yf is the solution to equation 
(SZD, i.e. for r and A given by (14811 and (j49p . For general (nonsymmetric) forcing terms F and 
G the corresponding process 1* can not be decomposed anymore as ytyf for 

sorne process yt- Due to this fact, it is not possible to apply a classical duality argument (as in [2^ 
or [2^) to extract some information for P and the corresponding BSDE. 


Remark 4. Note that in the case of cr Lipschitz (thus D^cr bounded) it is quite easy to derive the 
dissipativity of P in sense of (14611 . This particular case can be treated in the same way as in the 
section on first adjoint equation. 


7.1. Construction of Here we propose a different way to construct the process P, following 
ideas of Fuhrman et al. [7|. More precisely, will show that there exists r > 0 and a well defined 
matrix-valued process P such that the following duality relation holds 


E 


e-^* Tr 


DlH{XuUt,pt,qt)Yl 


dt 


d 

i=i ' 


e-^^(^Pt6a^iXt,ut),5a^{Xt,ut))dt + o{e) 


(51) 


Once we have this relation, it is easy to prove the stochastic maximum principle using usual argu¬ 
ments. The strategy to do so will be the following. 


Dual identity satisfied by P: For t > 0 and an arbitrary vector 7 G let us consider 
the following SDE 


dy^s'^ = Dxb{Xs,Us)ys'^ds + Yl1=i D^cr^iXs, Us)ys'^dwi, s >t, 


67 
Vt = 7- 


(52) 


By repeating the arguments by Yong and Zhou [^, Chapter 3, the SDE for the product y^''^ 
is of the form (with the notation At = Dxb{Xt,ut) and = D^cr^ {Xt,ut)) 


d U” (viX) = 


{y‘/Y + y<f (yiXY + E (Yf {BiY 


+ 


J = 1 


ds 


YBiyi”(y'.-T +!/ 7 (!' 7 ) (Bi) 

i=i 


dWl 


(53) 


Suppose for a moment that we are able to find a solution to equation (14511 in ^ (M_|_;<S") x 
(M+;iS"))'^ for some r > 0. Noting that {Psyt’'^,yl’'^) = Tr:{Psyl’'^{yl’'^)’^} and using equation 
dSSD, it follows by the Ito formula that for all [t,T] we have E-almost surely 


e-^*(Pir?,7)=E-^‘ (P^y^’^ 

= E-^‘ [e-’'^(Pry^^y^^)] +E^^ ds, 


( 54 ) 
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where we have used the notation D'^H{t) := Qt) for the forcing term in the equation 

for P. Since the processes P{-) and assumed to be in some appropriate exponentially- 

weighted spaces, there has to be a sequence of times {Tn)n>i with Tn yP -|-oo as n ^ -|-oo such that 
P-almost surely 


lim 

n—>-+oo 





= 0 . 


(55) 


Passing to the limit along the above sequence {Tn)n>i produces the following formal relation 


/ OO 

(56) 

which can be used to define the process Pt- Our aim is to show that the right hand side of ()56p 
actually defines a continuous bilinear form that can be used to prove (1511) without any reference to 
the second adjoint BSDE. 


Existence of P: The following estimates on {y\’^)s>t are crucial to prove continuity of the bilinear 
form. 

Proposition 4. Let rj G M” and assume that Hypotheses (H1)-(H4) hold. Then there is a unique 
solution {yt’^)s>t £ ^(M+;E’^) to the equation (l5^ for some r. Moreover, there exists a constant 

C > 0 such that for t > 0 and s >t 


supE-^‘ < C|r?|^ ] 

s>t 

and for all h > 0, 0 < t < t + h and s > t + h 


— a.s. 


(57) 


(58) 


Proof. Let us choose r > 2ci/2- The existence follows immediately by Theorem [2] and the proof of 
(f57|l it is a easy consequence of Lemma [T] with the additional requirement r > 2 max{ci/ 2 T 3 }- To 

prove the continuity property (f55|l let us denote Zg = — yl'^ then, fox s > t + h, we have by 

the Ito formula 

e-''"E|z,|^ = e-^('+'^)E|r/-2/*:;'J^-rE T 

J t-\-h 

-|-E / e~^'^\Zr\^ {Dxb{XT-,Ur)Zr, Zr) dr 

I t+h 


■f 

+ ^e/ , 

j=l t-\-h 


e {DxCf^ {Xr,Ur)Zr,Zr) dj 


3 = 
d 


Jt+h 


e '^'^\Zr\ {Dx(T\Xr,Ur)Zr, Zr) dr. 


Using the same estimate of Lemma [T] we end up with 

which we can control in the following form 


( 59 ) 


Ek-sSj‘ = E 


rt+h 


rt+h 


Dxh{Xr,Ur)y^4'^dT + ^ / Dx^J^Xr ,Ur)y^.;'^ dWf 


1 = 1 ' 


<CE \Dxb{Xr,Ur)y^^^fdT + 'r \Dxa\Xx,Ur)y^^^\UT. 

Jt Jt 
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Now, using Holder inequality and again Lemma [T] for the first term we obtain 

pt+h 

E \D,biXr,Ur)y^^^\UT 

, /•t+h ^ \ — / 

< (e e^^e-'^^\\D,b{Xr,Ur)\fdTy(E j' 

<h( sup (e-^"E|y*’’'|«))"( sup {e-^^K\\D,b{Xr,Ur)\f)' 

< Ch. 


( 60 ) 


The Dxcr term can be treated in the same way and the conclusion follows. 


□ 


Proposition 5. Let Hypotheses (H1)-(H5) hold and 7 , r/ G M”. Then there exist r > 0 and a 
progressive process {Pt)t>o with values in such that for all t > 0 it holds 


/ OO 

e-ris-t)^DlHis)yl’^,yl’'^)ds, F - a.s. 


(61) 


Moreover, sup^>oE||Pt |p < 00 and for e \ 0 we have that 


F\{{Pt+e- (62) 

Proof. For 7 and y G M"" fixed, let us define {Pt'y,r]) by the formula given in the statement. To do 
so we have chosen an arbitrary version of the conditional expectation. To construct the process Pt 
we have to prove that the map ( 7 , y) 1 —> {Pty, y) is a continuous bilinear form. Note that 


/ OO 

e-P^-t) {DlH{s)yl’\yl'^)ds 

/ OO 

e-Ps-t)\DlH{s)\\yi’^\\yl’^\ds 


< C\y\\y\ 




i/p 


J_ 

2 q 




(63) 


where we used conditional Holder inequality with p G (1, 2), g = > 2 and estimate (1571) . > 0 

can be chosen such that e~'^^F^*\D‘^H{s)f’ds < 00 . This can be seen from the definition of the 
Hamiltonian, the estimates on first adjoint processes and the polynomial growth of the coefficients. 
Further, there exists a set N such that F{N) = 0 and for u ^ N, 


\{Ptiw)y,'y)\ < C\y\\y\. 


If we set Pt{uj) = 0 for w G A^, by now we have constructed an adapted process Pt which satisfies 
equation (1611) . The symmetry of the process P is obtained easily by symmetry of D^H{s). 

To construct a progressive modification of Pt, it is sufficient to prove that the map (w, t) i-A Pt{uj) 
is J^( 8 >H(M+)\H(M"'^"')—measurable (i.e. it is a jointly measurable process). Here, H(M"^^"') stands 
for a Borel cr—field induced by the norm || • II 2 on If we prove that P is an (J7)~a'dapted 

and jointly measurable process then there is an (J^t)—progressive version of P. For a recent and 
elegant proof of this fact see 14]. Concerning joint measurability of P, its proof is given in [^. 
In that paper, in fact, even the existence of a progressive version in infinite dimensional setting is 
provided without any reference to the classical result. 
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To show that holds, let us write 

/ OO 


_l_ e-Ts+s 


/*t+£ 




' t-\-£ 
roo 


{{DlH{s)yi+^^\yi+^^^) - {DlH{s)y^^\y]:'^)) ds 


+ {DlH{s)yl’^, yl’^) ds. 

The first summand on the right hand side goes to zero in L^(n) as e \ 0 thanks to the Levy 
downward martingale convergence theorem (note that we have UC filtration (T't)t>o), the second 
one and the last one tend to zero in L^{Q) by dominated convergence theorem. Regarding the third 
term the result easily follows by using (I58|) . Indeed we can rewrite it as follows 


POO 

E-T+e / ^-r(s-t-e) 

Jt+e 

f 

+ E-^*+^ / 

Jt- 




—r(s—t—e) 


(64) 


e ^ -'{DlH{s)yl’^,yi+^’'r_ytn)ds = h + l 2 . 


> t-\-£ 


Using Holder inequality with p G (1, 2),q = > 2, the first addendum R can be estimate by 


' t-\-£ 


(65) 


g-rS]£.Ft+e|yl+£,7|29'\ 


Repeating the same estimate for the second addendum I 2 , using Lemma [U and (f58]l we get the 
required result. □ 

Remark 5. If F,G are random variables in L^(H) measurable with respect to T) then it is true 
that 

{PtF,G)=E^^ l\-^^^-^^(DlHis)yy,yl’^)ds, E - a.s. 

The proof follows by applying similar procedure as in Peng and Shi, [I^, Theorem 13. 

Proposition 6. Let {yt)t>o ® solution to the first variation equation (flTj ). Then there exists 
r > 0 such that the following relations hold true. 

i) e“’’<‘"+'>E( (P,„+j - Pto)yl,+„vl,+s) = o(0, 

ii) c-’'<‘"+'>E{P,.9U.<+,> 

^ POO 

= Ve / e-^\Ps5a^ {Xs,Us),5a^ {Xs,Us))ds + o{e). 

1=1 

Proof, (i) From Proposition [T]-(ii) we know that there exists r such that 

/ \ l/2k 

('e-Rto+p)E|£i/2y.^^j2A:^ < C, k>l, (66) 

and by the Markov inequality, for every <5 > 0 we have that 

E (|e^/^yfo+p| > 

If we denote Qs,e the event where is the centred ball with radius 

then it holds 




( 67 ) 
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Now we rewrite (i) in the following form 

e-Kto+.)]E< 

+ (^( {Pt,+e - Pto) 

=: +A|. 

The first term can be easily treated by the Holder inequality, Proposition [S] and estimates (| 66 |) . 
dSZD, respectively. Hence, the estimate reads 

• (e-Hto+.)]p (f^y (^8) 


Regarding the second term, we have that 


|H|| < 


sup 

T^^CT-1/4 


{{Pto+s 


Pto)x,x) 


Since Bq^-i/a is compact, there exist Ng open balls Bxi,s which cover it. Moreover, for all x € 
B(j^-i/a we can choose i such that \x — Xi\ < 5. Then 

{{Pto+e - Pto) x,x) = {{Pto+e - Pto) Xi,Xi) - {{Pto+s “ Pto) {x - Xi), {x - Xi)) 

+ 2{{Pt^+e-Pto)x,{x-Xi)) 

(691 

= ((-Pto+e - Pto) + \\Pto+e - -Ptolloo^^ 

+ 2||Pto+£-Ptoy|x|5, 


where, for a generic matrix T € we have used HTy := sup{|(Tx,y)| : x,y G M", |x| < 

1, 12 /I < !}• Taking supremum and expectation we obtain 

Ns 

|H|| < Y,mPto+e - Pto) Xi,x,)\ + c{5^ + 5P^) . (70) 

i=l 


If we let e ^ 0 and use (f62]) it follows that 

limsup|H|| < 0 ( 5 ^ + 
£->■0 


hence |j4|| + lA^l —t 0 , when d —> 0 and the proof of (i) is finished. 


(ii) Let us rewrite e ^(*o+^)E {PtoVto+ei Vto+e) ™ following form 

e-’'<'"+')E {«.»£+,.<+,> = E [n- |p,„ (e+l‘«+'><+.) (e-§<‘"+'>!,f.+,)' 

Thanks to the ltd formula and equation (|47l) . we obtain 

d (e-^^y/) = [-ry/ + Dxb{X,,Us)Y^ + Dxb{Xs,Us)^ 

d 

+ ^ Dxa\Xs, Us)Y,^Dxa^{Xs,Us)^ + r{s)] ds 

i=i 

d 


+ E'- 

i=i 


Dxa^(Xs,Us)Y^ + Y^Dxa\Xs, + A\s) 


dWi, 


(71) 


where we have used the notation y/ = yl (yf)^ and r(s), Al(s) are as in (|45]) . ()3U|1 . Now, by taking 
conditional expectation with respect to Pto ^-nd rewriting the equation in integral form from to to 
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s (recall Remark [5]) it follows that 


E-^‘0 (e-^*y/) = E-^‘0 r \-rY^ + D^b{Xr, Ur)Y,^ + Y^D^b{Xr,Urf' 

Jto 


dr 


d „g 

+ ^E-^*o / 
i=i 


D^a^{X^,Ur)Y,^D^a^{Xr,Urf+ r{T) 


dr. 


Hence, taking into account the definition of F in (HHI) . multiplying by Pt^, setting s = Iq + e and 
taking expectation, we arrive at 




no 


e-’^^E [Tr {Pt, {-rY^ + D^b{Xr,Ur)Y^^ + Y^ D^b{Xr,UrY )}] dr 


I'to+e 


j=lPo 


+ E 


cto-Ye 


3=1 


+ E 


(‘to-\-€ 


j=l 


e-""E [Tr{Pt„ [D^a^Xr^nYY^D^a^Xr^Urf)]] dr 
^Pto {6a\Xr,Ur) {6aYXr,Ur))'^^'^ dr 
j-Pto (^D^aYXr,Ur)yr {6a^ {Xr, Ur))^'j'j 


e“^^E 


Tr 


e“''^E 


Tr 


dr 


+ Y, e-’'"E [Tr {Pt, {5aYXr,Ur){y^,f D^a^X^, u^f )}] dr. 

3=1 

We will show that using the estimate for yf in the form of (16611 , the only term which is not of order 
o(e) is the third one. Therefore, the final equality will read 

^ ptQ~\~S 

= ^E / e~^'^(Pto6aYXr,Ur),6aYXT-,UT)\dT + o{e). 

3=1 

For sake of completeness, let us estimate the second term as 

^ ptQ-\-e _ _ 

^ / e-'-'E [Tr {Pt, {D^a^X^, u^Y^D^a^Xr, Ur ?)}] dr 


(72) 


i=i 


nQ-\-£ 


^ / e '’'^E{PtgD^aYXr,Ur)yYDx(TYXT,Ur)yr)d3 


3=1 


< 


E 


cto-Ye 


e-^’^E 


i=i 


< 


E 


f'to-\-£ 


j=i 


\PtJYDxa^iXr,Ur)\^m^ 

T/2 


dr 


1/2 _ 


IE(e-’'"||Ptoll 2 ) ' E[e-^YDxaYXr,UrT)’ E[e-''YyY^)’ dr, 


1/2 


and the order of o(e) now follows by Proposition [5l the polynomial growth of DxCr{-) and estimate 
dMD, respectively. The remaining terms can be treated in the similar way. 

To finalize the proof of (I72p . it remains to be shown that 


E>^ 

3=1 ■ 


rto-Ys 


to 


e {Pr - Pto) dcrYXT,Ur),SaYXT,Ur))dT = o{e) 


(73) 


But this is easily obtained by repeating the same arguments as in the proof of (i). The proof of 
the Proposition is now concluded. □ 
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8. Necessary stochastic maximum principle 

For our main result we need to recall the notion of the Hamiltonian of the system. Given the 
control problem ©-([S]), let us define H :W^ x U x x —>• M as 

H{x,u,p,q) = {p,b{x,u)) + TT[q^a{x,u)] - f{x,u). (74) 

Theorem 5. Assume (H1)-(H5) hold and let {X,u) he an optimal pair. Then there exist r > 0, a 
pair (p,q) € x and a progressively measurable process {Pt)t>o with 

values in <S” such that the following variational inequality holds, P 0 dt—a.e. 

H{Xt,v,pt,qt) - H{Xt,Ut,Pt,qt) + 2 - a^{Xt,Ut)) ,cr^{Xt,v) - (j\Xt,Ut)^ < 0, 

for every v € U. The pair of processes {p, q) is the unique solution to the first adjoint equation (fSSll . 
The definition of the process Pt is given in Proposition\^ and the process satisfies supj>oE||PtII 2 < 
00. 


Remark 6 . A sufficient condition for such r is given in the Appendix. 


Before proving the theorem, it is useful to rewrite the variation of cost functional in a suitable 
form, as the following proposition suggests. 


Proposition 7. There exists r > 0 such that the following expansion holds 

d 

^ {ql,5a^Xt,ut)) - {pt,db{Xt,ut)) + 6f{Xt,ut) 


J(n^(-))-J(h(.))=E 




1 r 

-E / 

2 Jo 


j=i 


dt 


(75) 




Tr 


DlH{Xt,ut,pt,qt)yt (y!) dt + o{e), 


where H{Xt,ut,pt,qt) is the Hamiltonian of the system computed along the optimal trajectory. 
Proof. See Appendix. 

Now we are in position to end the proof of the SMP. 

Proof of Theorem 0 . The difficult step of the proof is to show that the following holds 

j-OO 

e-^^DlH{s)yl,yl)ds = Y,^ / e" 

i=i 

Indeed, if relation (f76|) holds true then by using Proposition [7] we get 

0< J(n^(-))-d(h(-)) 

d 

{Xs,Us)) - {ps,db{Xs,Us)) + Sf{Xs,Us) 


□ 


E 


f 


Ps5a\Xs,Us),5a^ {Xs,Us))ds + o{e). (76) 


= E 


POO 

/ 

^0 

-I ^ PC 

i.i •'« 


i=i 


dt 


(77) 


Ps6a\Xs,Us),6a\Xs,Us))ds + o(e), 


thanks to the optimality of u{-). Now the final variational inequality follows by standard arguments, 
i.e. by using the definition of 5a^,5b,6f, noting that = [toPo + £] and by sending e \ 0 . 

Let us focus on the proof of (|76ll . Recalling Remark [2l we can rewrite the left hand side of (1761) in 
the following form 


E / e-^^{DlH{s)ylyt)ds 

rto-\-€ 


JO 
= E 

= E 


ptQ-\-£ pGO 

/ e"’’* (p 2 p(s)yf,yf)ds + E / e-'^^DlH{s)yl,yl) ds 

J to j tQ-\-£ 

Jtc 


tQ-\-£ 
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where we have used Proposition [T] to estimate the first integral and the identity yf = Us 
for s > to + £• Taking into account Remark [Sj we finally deduce the following decomposition 

poo 

E / {DlH[s)yl,yl) ds = + o{e) 

Jo 

= (P,„+, - P„) 

Finally, by Proposition El the proof of the Theorem is now concluded. 


□ 
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10. APPENDIX 

10.1. Proof of Proposition [H 

Proof. In the following we are going to linearize the equations satisfied by and C''(‘) ia 

order to use the estimate obtained in Lemma [H 

(i) It is easy to see that the equation for ^^(•) can be rewritten in the form 

d 

det = [G,m + 5btXEAt)\dt + Y.[Gim + 5aixEAt)]dWl, 

i=i 

where 

G'fe(t) := D,h{Xt + e^t, ut)de, Giit) := D,a^{Xt + uDdO. 

Jo Jo 

Thanks to Hypothesis (H4) we can apply Lemma [T] and obtain (the constant K > 0 varies from 
line to line) 


sup e-^^'E|efr < K 
teR+ 


f 


--t 

e P 


' txEAt ) n"dt 


1 2k 


+ K 


a 

n. 


^—rt 


j=l 




1 k 


< K 


I 


--t 

e ^ 


't,ul) - b{Xt,ut)\^^] dt 


2k 


-rt l-!u>\X t V ,.e\ ~i(Y. r, df ^ 


+ ^ E f / (Xt, ) - ai (Xt, ut)\ 

i=i 

< K[e^^ +s’^] < iFe^ 


(78) 


thanks to the polynomial growth of the coefficients and the boundedness of the integration interval 
E^. Indeed, remember that it is easy to control all the moments of X up to a fixed time. In this 
case the discount factor pi can be chosen equal to the initial one pi = r. 


(ii) Using again the global monotonicity assumption and Lemma (H the estimate for y'^ follows 
in the same way. 

(iii) For we start by estimating its norm in the space M”), for a generic a G M. 
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Using the same technique as in Lemma [U we obtain 


—rkati] 


sup e 
tGR+ 


/ 


_ ra 

e 2 


( 1 \ -\2k 

mtXEAt) + -DlbiXt,ut)iytf\^^ ] dt 






—rat 


•(^l)XEe{t)yt + ^Dla\Xt,ut){ylf\'^^y dt . 


The first term (with 5bt) can be treated as before, thanks to the boundedness of E^. Let us discuss 
the second one. It holds 


(E\Dlb{X„u,){y, 

Jo 


.e\2\2k\ 2k 


dt 


e —nDibiXt,ut)\ 
1 


— \|4A:\4fc (^—raktTr7\^,e\Sk\4:k 


m 


dt 


—rakt 


t>0 


m 


Sk 


< ( SUpe 

^—raktju'l^.e 


4fc 


roo 


10 


e-"ri (]K\Dlb{Xt,ut)\^’') '' dt 


< K (sup e 

,t>o 


Vt 


< K ( supe-^'^^Elyf 

.t>0 


8 /c 


\Sk 


f 


-mt 


e 4 




|2m+l |4fc \ 4fe 


< K (sup e 

J>0 




uo 

1 r 


/*CX) PC 

/ e-'^^dt + 

Jo Jo 


_ ra j. 

e 4 


dt 


|4fc(2m+l)\ 4fc 


f 


e i^dt -\- 


I 


dt 


4k 


where we used the polynomial growth of D^b and Jensen inequality, assuming that e i^dt < oo, 
hence ra > 0. Moreover, if we choose a > max (4, 8k{2m + 1)) = 8k{2m + 1), we have that 


(K\Dlb{Xt,ut){y\ 


e\2\2k \ 2k 


dt < Ke, 


(79) 


where K = K{r,k,m) depends on the choice of the initial discount factor, the order of integra¬ 
tion and the polynomial growth of the coefficients of the state. Let us briefly sketch also the 
computations for the last addendum 


[nDla\Xt,ut)iy: 


e\2\2k\ k 


dt 


< K { supe-™^'E|?/f|®^ 

.t>0 


< K ( supe-"“^‘E|yf'^'' 

J>0 


\^ + \Xt 

If a > Akm, following the same strategy as above we end up with 


/“e-T‘ (E|Oj<rJ(X,.S,)|")” dt 

f 


_ ra 4 . 

e 2 


m|4fc\ 2k 




£\2|2fc\ k 


dt < Ke^ 


(80) 


Summing up all the estimates and using Lemma [H we easily get the required result, for some pa 
big enough. In this case it is sufficient to choose ps > ar > 8k{2m -\- l)r. 

(iv) Following Yong and Zhou 2j], it is easy to see that 


where 


a 

drjf = [D^b{Xt,ut)rjt + Af] dt + ^ ut)pl + -Bj’' 


dWl, 


i=i 


Zlf := dbtXEM + [Gb{t) - D^b{Xt,ut)] Ct; 

Sr := {Gi{t)-Dia{Xt,nt))ef 
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Let US consider A^{-) first. 


f 


_ roe j. 

e 2 




<K 

Jo 

+ K 


'tXEAt)?l "dt 


_ roe ^ 

e 2 


10 


(E\[Gb{t) - D,b{Xt,ut))]etr) "’^dt 


<Ke + K { supe-™*^‘E|.ef 1^^ 


t>o 




e 4 


- D,b{Xt,ut))f^]"'^ dt 


<Ke + 


e-^* (E|G6(t) - dt, 


due to the previous result with a >2, and Holder inequality. Regarding the last term we have 

Gbit) - D,b{Xt, ut)) = [' [D.biXt + O^t, ut) - D,biXt,ut))] dO 

Jo 

= [ [D,biXt + eet,ut)-D,biXt + 9et,ut)]de 

Jo 

+ [ [D,,biXt + Oil Ut) - Da,b{Xt,ut))] de. 

Jo 

Hence, using the Taylor expansion with Lagrange rest, there exists x (depending on t and ui) such 
that 


poo 

J e~^*\Gbit) - D^b{Xt,ut)\^4k^Q)dt 


< / ( E 


+ / e--* ( E 


D,,b{Xt + Oil ul) - D,,biXt + Oil ^t)] dO 


Ak\ 4 fe 


dt 


I 


"I 4 A:\ 4 fc 


D'^bix, Ut)0i^it)d9 


dt 


<Ke+ (supe-™^*E|ef|®^ 
\t>o 

<Kie + e^ll, 


_ roe 4 . 

e 8 


(E\Dlbix,ut)f^^ "''dt 


thanks to the estimate obtained in point (i) and the polynomial growth of D^b (here we have to 
require a > 32k{2m + 1)). Then 


' (E|Hf |2") dt<Ke + Ke^/‘^ H 






< Ke. 


For Bl^, proceeding in a similar way we obtain 


f 


—rat 


{t)-D^a^iXt,ut)\^’‘) ""dtKKe. 


To conclude, we apply Lemma [T] to get 

( noo , s m \ ^ / noo i \ ^ 


< K{£‘^^+£^^) = Oi£^^). 

In this case can be chosen as P 4 > ra > 32A:(2m + l)r. 
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(v) Let US denote = d{r]^{t) — C^{t)), 

f dQ = {D,b{Xt, ut)Q + Af) dt + ZU {Dxcr^iXt, ut)^ + dW^, 

lC(0) =0, 

where 

:= SD^btXEAm + ^ [Gbit) - DlbiXt, uf)] 

+ \dDlbtXEMiet)^ + \Dlb{X^,umif - iftfl 


Bf := 6D^atXEeit)v! + \ [Gait) - DlaiXt,ul)] 

+ ^dDlatXEeit)i^!)^ + ^DlaiXt,ut)[iCt)‘^ - (yf)^], 
d 

I Gbit) := 2 f; eolbiext + (1 - e)xf,u!)de, 

:= 2/; eDlaieXt + (1 - 

First, let us consider the A^i-) term. Applying the Holder inequality gives 


_ rot j. 

e 2 


< 


£12?;:^ 2fc 


dt 


e A 


in5D^btXEAm?'')^^ + \in[Gbit)-Dibim,u^^^^^ 

dt 


2\2k\ Oh 


+ t{E\sD%xEAm(i)T)* + - fenh"*)*' 

t>0 JEe 

poo __ 

+ A(supe-™^*E|^f|®^)« / e-'f^iElGbit) - DlbiXt,ul)\^'")^dt 
t>o Jo 

+ A(supe-™^*E|^f|®^)^ [ e-"r^iE\5Dlbt\^’')^dt 

t>o Jes 


Sk\^ 


+ A(supe-™^*E|r?f (supe-™^*E|ef + yfr) 

t>0 t>0 


e-—{E\DlbiXt,ut)r)^^dt. 


If a > 4 the first and the third term con be easily controlled. For the last addendum we use the 
same technique as in (iii) to get the boundedness of the integral for a > 8A:(2m + 1), hence 


_ rot i 

e 2 


£|2fc\ 2fc 


dt<K 


£3/2+ e 


e-^* iE\Gbit) - DlbiXt,ut)\‘^’^) dt + e^ + 


Finally, we can rewrite E|Gfe(t) — x(t),in the following form 

Gbit) - DlbiXt.ul) = 

= 2 [ ODlbiext + (1 - e)xi,ui)de - DlbiXt, ui) 

Jo 


(81) 


= 2 9 

Jo 


Dibiext + (1 - e)xf,ui) - DibiXuu\ 


mo- 


If a > 8/c(2m+l), by the continuity of the map x i-A D^6(x, u) and dominated convergence theorem 
it follows that 


-mt 


e 4 


Dlbiext + (1 - e)xi ui) - DlbiXt, ul) 


dO dt —> 0, 
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as e ^ 0. Then 

dt < K[£^/‘^ + + o(e) = o(e). 

For B^{t) we proceed in the same way to obtain 

e-^ (E\B^’^{t)f^y dt = oie'^). 

Using Lemma [U the desired result follows for P 5 > ra > 8k(2m + l)r. □ 




10.2. Proof of Proposition [S]. 

Proof. Since Yf = the existence and uniqueness of a solution follow from the existence 

and uniqueness of the process (see Theorem [2]), with the restriction r > 2 ci/ 2 - 

Let us now denote At := Dxb{Xt,ut), Bj := DxCr^{Xt,ut) and note that a symmetric (positive) 
definite matrix Yf can be decomposed as Yf = ^ where 7 i > 0 for all i and (cj)i is an 

orthonormal basis of M”. Clearly, each yj and c* depend both on t and e but we omit this notation 
in the proof. 

Having in mind the above, one arrives at 

n n 

{AtYf,Yt% = TtiAtYfiYff} = TT{Atc,cJc,cf} = Mc^c[ Ac^cJ} 

i=l i=l 

n n 

= ^ li {Aci, Ci) Tr{cicf} = ^ {AtCi, a ), 

^=1 ^=1 


n 

{BlYABlf, Yt% = Tr {B^YfiBlf {Yff } = ^ {B{fcAl} 


i=l 




iCi)?\cA 


2=1 

n 


2=1 




2=1 


2=1 


and 


\BiYf\\l = TV [BiYf {Yff [Bif] = 7 ' TV [bIc^cJ acj {BD^ 

2=1 

n n 

= {cicf {BlfB^acJ} = ((BifBlci, a) 


2=1 


2=1 




2=1 

where we have used the basic properties of the Trace. Using these estimates we are able to prove 
the following dissipativity condition 

d d 

{AtYf + YfAj, Yf) + Y {BiYfiBlf, Yf) + YW^lYt + II2 

i=i j=i 

n d n d n 

< 2 ^ 7*^ Ci, Ci) + ^ ^ I (^i )^Ci IV 2 ^ ^ 7^ I Bla I^ 

2=1 j = l 2=1 j = l 2 = 1 


(^tCi,Ci) + 


2=1 


i = l 


<2c3/2^7'|Cip = 2c3/2r/||^. 
2=1 
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Now, applying the ltd formula to e ^*111^*^112 we obtain, for every T > 0 and <5 > 0 


Jo 


e\\2 
t \\2 


dt 


= 2IE / e 


/ 


^—rt 


(y/, + YlA^) + £ (y/, BiYfiBtf) 


i=i 


dt 


+ 2E / e 


{Yf,Tt)dt + ^E [ e-^^\\BiY,^ + YfBi+Al\\ldt 
JO Jo 

rT pT 

<2c 3/2E/ e-^*||y/||^dt + <5E / e-^*||y/||2dt 

^0 Jo 


d 


+ 


tw.! e-^<\\I,\\ldt + '£l e-^<\\4\\idt 
° Jo Jo 


Hence 


d p.j' 


{r-2c,/,-6)E [ e-^^\\Yf\\ldt<^E [ e-^^\\T,\\ldt+ ^ 

Jo 0 Jo Jo 

and the estimate follows for r > 2c^/2 by sending T +oo. The final estimate holds for r > 

2max{ci/2,C3/2}. □ 

10.3. Proof of Proposition [T]. 

Proof. If we compute the Ito differential of the processes {iJuPt), where pt is the solution to 
the finite horizon equation (1371) . we obtain 

d 

iVt^Pt) ) = {d{e~''^yl),pt) + e"’’* {yt,dpt) + ^ {q{,D^a\Xt,ut)yl + 5(T^{Xt,ut)) dt 

j=i 

= [ - re"’'* {y!,Pt) + e"’’* {D^b{Xt, ut)yt,Pt) - e"’’* {yf, D^b{Xt,utfyf) 

d 

- E D^a\Xt,utfq>) + e"’’* (yf, D^f{Xt, ut)) + re"’’* {yt,pt) 

d 


+ E i<ll^Y)xCj\Xt,ut)yl + 5a^{Xt,ut)) 
i=i 


dt + MtdWt, 


where the stochastic term is a local martingale with zero mean value (which can be proved by 
standard localization argument). Hence, by taking expectation we have for all T > 0 

e"’'^E(yf,,pr)-E / {yl,D^f{Xt,ut)) dt = '^E f e~'^^ {Xt,ut)) dt, 

Jo Jo 

thanks to the fact that Pq = 0. Since {pt)t>o £ M"') then there exists a sequence of times 

{Tn)n>i with Tn +00 as u —> +00 such that along this sequence E —> 0. Hence, for 

all n € N we have that 


E(e 


-rTr 


J/t„,PtJ-E f e {yf,D^f{Xt,ut))dt = '^E f e {ql,6a^ {Xt,ut)) 
Jo Jo 


dt. 


Thanks to the growth assumptions on a, f and to the regularity of yf and qt, we can send Tn to 
infinity to end with 


E 


noo ^ roo 

/ e-^^{yt,D,f{Xt,ut))dt = -^E e"’’* (g^ n*)) 

Jo Jo 


dt. 
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Repeating the same argument for e {zf,pt) we get 

{z!,pt) ) = {d{e-^h!),pt) + {zt, dpt) 

+ {qt^D^a\Xt,ut)zf + Sa^{Xt,ut)yl + -Dla^yff) dt 


i=i 

- re"'’* {zf,pt) + e”'’* {D^b{Xt,ut)zf ,pt) + e”'’* {6b{Xt,ut),Pt) 
+ {Dlb{X,,ut){yt)\pt) - {zf, D,b{Xt,utfyt) 

d 

- Y, e-"* (4, D^a\Xt,utfq>) + (4, ut)) 

i=i 

d 

+ re“''* (4,pt) + ^ {Xt,ut)zl) 

i=i 

d 


+ X] iqi^^(^^i^t,ut)yt + -Dla^{ylf) 

i=i 


dt + iVtdTVt, 


where the stochastic term is a local martingale with zero mean value (which can be proved by same 
argument as before). Hence, taking expectation we obtain for all T > 0 


-E / e-'^^{zlD^f{XuUt))dt = ¥. / {5b{XuUt) +-Dib{Xt,ut){yt)\pt) dt 

.In .In ^ 


i=i 


The term o(e) comes from the following estimate 


+ ® + o(e)- 

•_1 t/ 0 




<CE e-^\l + \XtrM\\ql\dt 


<C (E 

Je, 


e-^\l + \Xtrf\yt'^ 


1/2 




1/2 


dt 


< Csup [Ee-^^y: 

teE, 


1 1 


1/4 


Ee 

1/2 


Ee-^*|g^|2 


1/2 


dt 


<Ce(^J^ Ee-^^\qifd?j , 


and the last integral goes to zero as e goes to zero, since E e~'^^\ql\^dt < oo. Applying the same 
strategy as before we can choose a sequence (T„)„>i with T„ ^ +oo as n ^ +oo such that along 
this sequence E[e“^^"PT^] —)■ 0. This way we end up with 


-E e {zl,D^f{Xt,ut)) dt 


= E 


f>oo _ 1 _ 

e"’’* {6b{Xt,ut) + -Dlb{Xt,ut){ylf ,pt) dt 



{Dla^{Xt,ut){ylf, qt) dt + o(e). 
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If we substitute relations (I52|) and (I55|l into equation (1551) . we obtain 

d 


JK(-))- 


))-j(u(.))=E r 
^0 




'^{ql,5cr^{Xt,Ut)) - {pt,Sb{Xt,Ut)) + 5f{Xt,Ut) 


i=i 


dt 


+ -E 
2 


f 


^ — Tt 


'^{Dla^{Xt,ut){yff,qt) - {Dlh{Xt,ut){ylf ,pt) 
i=i 


+ {DlfiXt,ut)yt,yt) 


dt + o(e), 


and recalling the definition of the Hamiltonian u,p, g) = {p,b{x,u))+T t [q'^a{x,u)] — f{x,u) 
we have the desired result. □ 


10.4. Conditions on the disconnt factor r. Here we collect some restrictions on the discount 
factor used throughout the computations in the paper. For the purposes of the SMP it is not 
necessary to find precise values of the discount factor, in general r has to be positive and big 
enough. Nevertheless, it can be useful to exhibit some sufficient conditions. 

Starting from the well posedness of the state equation, we have to require r > 2 ci /2 in order 
to find a unique solution in the space M*^). Regarding the first variation equation we 

have no other restriction. On the contrary, to assure that D‘^b{Xt,ut){yt)‘^, D'^a^{Xt,ut){yi)‘^ € 
in the equation for z^, a sufficient condition is r > 2 max{ 0 , Ci/ 2 , C 2 ( 2 m+i)-i) C 2 m-i) C 3 }, 
where C 3 comes from estimate (11811 applied to the process y^. Further restrictions come from the 
proof of Proposition [TJ Here it follows that one can choose pi,p 2 > 2 ci/ 2 ; P 3 ,P 5 > lQk{2m + 
l)max{c2A;(2m+i)-i,C8fc-i,C2fcm-i} and Pi > 64k{2m + 1) max{c 4 fc( 2 m+i)-i, C 4 fcm-i}- These condi¬ 
tions are derived from the polynomial growth assumptions and from the use of the Holder inequality. 

The choice of the discount factor r for the first adjoint equation (see Theorem [4]) depends on the 
a priori estimate given by Lemma [3] as well as the integrability of the forcing term Dxf{Xt,ut). 
Therefore, due to the polynomial growth, it is easy to see that it is sufficient to consider r > 
2 max{ 0 ,Ci/ 2 ,Q_i}. 

For the existence and uniqueness of {yt’^), we choose r > 2c\j2- Regarding the estimates (l571l 
and (1551) . it is sufficient to choose r > 2 max{c 4 / 2 ) C 3 }. Now, for the existence of the process P, it 
is sufficient to take r > 2max{0, C5, C3(2m+i)-i)csm-i, C3;_i} (for p = = 3 in (1551) 1. Regarding 

Proposition [51 we have to add some restrictions originating from Lemma [T] throughout the proof. 
More precisely, it is sufficient to require r > 2maxjcy, C2m-i,C3}. 

To conclude, the statement of the SMP holds true if the discount factor is chosen in a way such 
that all the previous results can be applied. Hence, it is sufficient to choose k = 1 and r such that r > 
64(2m-M) max{0, Cxl 2 XZ^ £ 5 , 07 , Q_i, C 3 /_i, C 2 m- 1 , CSm-l, Qm-l, C 2 ( 2 m+ 1 )-D C 3 ( 2 m+ 1 )-D C 4 ( 2 m+l)-l}- 
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